Spatiotemporal Saliency and Background Subtraction in Dynamic Scenes

A background subtraction algorithm, based on center-surround saliency, is proposed. Background subtraction is formulated as the complement of saliency detection, by classifying non-salient (with respect to appearance and motion dynamics) points in the visual field as background. The algorithm is inspired by biological mechanisms of motion-based perceptual grouping, and extends a discriminant formulation of center-surround saliency previously proposed for static imagery. Under this formulation, the saliency of a location is equated to the power of a pre-defined set of features to discriminate between the visual stimuli on a center and a surround window, centered at that location. The features are spatiotemporal video patches, and are modeled as dynamic textures, to achieve a principled joint characterization of the spatial and temporal components of saliency. The combination of discriminant center-surround saliency with the modeling power of dynamic textures yields a robust, versatile, and fully unsupervised background subtraction algorithm, applicable to scenes with highly dynamic backgrounds and moving cameras. The algorithm is tested on challenging sequences, and shown to substantially outperform various state of the art background subtraction techniques. Quantitatively, its average error rate is almost half that of the closest competitor.